Mirror mirror on the wall... an unobtrusive intelligent multisensory mirror for well-being status self-assessment and visualization

A person’s well-being status is reflected by their face through a combination of facial expressions and physical signs. The SEMEOTICONS project translates the semeiotic code of the human face into measurements and computational descriptors that are automatically extracted from images, videos and 3D scans of the face. SEMEOTICONS developed a multisensory platform in the form of a smart mirror to identify signs related to cardio-metabolic risk. The aim was to enable users to self-monitor their well-being status over time and guide them to improve their lifestyle. Significant scientific and technological challenges have been addressed to build the multisensory mirror, from touchless data acquisition, to real-time processing and integration of multimodal data

Exploiting scale-invariance: a top layer targeted inverse model for hyperspectral images of wounds

Detection of re-epithelialization in wound healing is important, but challenging. Hyperspectral imaging can be used for non-destructive characterization, but efficient techniques are needed to extract and interpret the information. An inverse photon transport model suitable for characterization of re-epithelialization is validated and explored in this study. It exploits scale-invariance to enable fitting of the epidermal skin layer only. Monte Carlo simulations indicate that the fitted layer transmittance and reflectance spectra are unique, and that there exists an infinite number of coupled parameter solutions. The method is used to explain the optical behavior of and detect re-epithelialization in an in vitro wound model

Real-Time Noise Removal for Line-Scanning Hyperspectral Devices Using a Minimum Noise Fraction-Based Approach

Processing line-by-line and in real-time can be convenient for some applications of line-scanning hyperspectral imaging technology. Some types of processing, like inverse modeling and spectral analysis, can be sensitive to noise. The MNF (minimum noise fraction) transform provides suitable denoising performance, but requires full image availability for the estimation of image and noise statistics. In this work, a modified algorithm is proposed. Incrementally-updated statistics enables the algorithm to denoise the image line-by-line. The denoising performance has been compared to conventional MNF and found to be equal. With a satisfying denoising performance and real-time implementation, the developed algorithm can denoise line-scanned hyperspectral images in real-time. The elimination of waiting time before denoised data are available is an important step towards real-time visualization of processed hyperspectral data. The source code can be found at http://www.github.com/ntnu-bioopt/mnf. This includes an implementation of conventional MNF denoising

A random forest-based method for selection of regions of interest in hyperspectral images of ex vivo human skin

Hyperspectral imaging is a useful tool for characterization of human tissue. However, the vast amount of data created makes it challenging and tedious to manually select spatial regions of interest for further processing. In this study, a random forest-based method was evaluated on basis of its ability to segment human skin regions from the background. The method was compared to the performance of two alternative methods, spectral angle mapper (SAM) and a K-means clustering-based method. The methods were tested on hyperspectral images of ex vivo and in vivo human skin in the wavelength range 400-1000 nm. The random forest approach was found to be robust and perform well regardless of image type. The method is simple to train, and requires minimal parameter tuning for good skin segmentation results

Real-Time Noise Removal for Line-Scanning Hyperspectral Devices Using a Minimum Noise Fraction-Based Approach

Processing line-by-line and in real-time can be convenient for some applications of line-scanning hyperspectral imaging technology. Some types of processing, like inverse modeling and spectral analysis, can be sensitive to noise. The MNF (minimum noise fraction) transform provides suitable denoising performance, but requires full image availability for the estimation of image and noise statistics. In this work, a modified algorithm is proposed. Incrementally-updated statistics enables the algorithm to denoise the image line-by-line. The denoising performance has been compared to conventional MNF and found to be equal. With a satisfying denoising performance and real-time implementation, the developed algorithm can denoise line-scanned hyperspectral images in real-time. The elimination of waiting time before denoised data are available is an important step towards real-time visualization of processed hyperspectral data. The source code can be found at http://www.github.com/ntnu-bioopt/mnf. This includes an implementation of conventional MNF denoising

Estimation of skin optical parameters for real-time hyperspectral imaging applications

Hyperspectral imaging combines high spectral and spatial resolution in one modality. This imaging technique is a promising tool for objective medical diagnostics. However, to be attractive in a clinical setting, the technique needs to be fast and accurate. Hyperspectral imaging can be used to analyze tissue properties using spectroscopic methods, and is thus useful as a general purpose diagnostic tool. We combine an analytic diffusion model for photon transport with real-time analysis of the hyperspectral images. This is achieved by parallelizing the inverse photon transport model on a graphics processing unit to yield optical parameters from diffuse reflectance spectra. The validity of this approach was verified by Monte Carlo simulations. Hyperspectral images of human skin in the wavelength range 400–1000 nm, with a spectral resolution of 3.6 nm and 1600 pixels across the field of view (Hyspex VNIR-1600), were used to develop the presented approach. The implemented algorithm was found to output optical properties at a speed of 3.5 ms per line of image data. The presented method is thus capable of meeting the defined real-time requirement, which was 30 ms per line of data.The algorithm is a proof of principle, which will be further developed

Hyperspectral characterization of re‐epithelialization in an in vitro wound model

In vitro wound models are useful for research on wound re‐epithelialization. Hyperspectral imaging represents a non‐destructive alternative to histology analysis for detection of re‐epithelialization. This study aims to characterize the main optical behavior of a wound model in order to enable development of detection algorithms. K‐Means clustering and agglomerative analysis were used to group spatial regions based on the spectral behavior, and an inverse photon transport model was used to explain differences in optical properties. Six samples of the wound model were prepared from human tissue and followed over 22 days. Re‐epithelialization occurred at a mean rate of 0.24 mm2/day after day 8 to 10. Suppression of wound spectral features was the main feature characterizing re‐epithelialized and intact tissue. Modeling the photon transport through a diffuse layer placed on top of wound tissue properties reproduced the spectral behavior. The missing top layer represented by wounds is thus optically detectable using hyperspectral imagin

Combined 3D model acquisition and autofocus tracking system for hyperspectral line-scanning devices

Obtaining a sharp and focused image is essential to fully utilize the advantages of hyperspectral imaging. For line scanning hyperspectral devices, focusing is a challenge in applications where the height and/or position of the imaged object might vary during a scan. Initial focusing is in addition a tedious process that has to be repeated for each sample or measurement. In this paper, a new continuous autofocus tracking system for hyperspectral line-scanning cameras is reported. The presented system is able to automatically and objectively find the correct distance between the camera and the imaged object for proper focus, and retain this focus distance during scan using a laser triangulation system. Concurrent with the focusing, a 3D model of the object is constructed. The system was tested and found to perform well for NIR-SWIR imaging of human hands and test objects with sharp changes in height and contrast. The method is easily adaptable to other spectral ranges and applications, such as industrial conveyor belt applications. The method significantly eases the acquisition of hyperspectral images by ensuring optimal image quality in every scan and eliminating the need for manual refocusing between individual samples

Wavelet based feature extraction and visualization in hyperspectral tissue characterization

Hyperspectral images of tissue contain extensive and complex information relevant for clinical applications. In this work, wavelet decomposition is explored for feature extraction from such data. Wavelet methods are simple and computationally effective, and can be implemented in real-time. The aim of this study was to correlate results from wavelet decomposition in the spectral domain with physical parameters (tissue oxygenation, blood and melanin content). Wavelet decomposition was tested on Monte Carlo simulations, measurements of a tissue phantom and hyperspectral data from a human volunteer during an occlusion experiment. Reflectance spectra were decomposed, and the coefficients were correlated to tissue parameters. This approach was used to identify wavelet components that can be utilized to map levels of blood, melanin and oxygen saturation. The results show a significant correlation (p <0.02) between the chosen tissue parameters and the selected wavelet components. The tissue parameters could be mapped using a subset of the calculated components due to redundancy in spectral information. Vessel structures are well visualized. Wavelet analysis appears as a promising tool for extraction of spectral features in skin. Future studies will aim at developing quantitative mapping of optical properties based on wavelet decomposition.© 2014 Optical Society of Americ

Hyperspectral imaging for detection of cholesterol in human skin

Hypercholesterolemia is characterized by high levels of cholesterol in the blood and is associated with an increased risk of atherosclerosis and coronary heart disease. Early detection of hypercholesterolemia is necessary to prevent onset and progress of cardiovascular disease. Optical imaging techniques might have a potential for early diagnosis and monitoring of hypercholesterolemia. In this study, hyperspectral imaging was investigated for this application. The main aim of the study was to identify spectral and spatial characteristics that can aid identification of hypercholesterolemia in facial skin. The first part of the study involved a numerical simulation of human skin affected by hypercholesterolemia. A literature survey was performed to identify characteristic morphological and physiological parameters. Realistic models were prepared and Monte Carlo simulations were performed to obtain hyperspectral images. Based on the simulations optimal wavelength regions for differentiation between normal and cholesterol rich skin were identified. Minimum Noise Fraction transformation (MNF) was used for analysis. In the second part of the study, the simulations were verified by a clinical study involving volunteers with elevated and normal levels of cholesterol. The faces of the volunteers were scanned by a hyperspectral camera covering the spectral range between 400 nm and 720 nm, and characteristic spectral features of the affected skin were identified. Processing of the images was done after conversion to reflectance and masking of the images. The identified features were compared to the known cholesterol levels of the subjects. The results of this study demonstrate that hyperspectral imaging of facial skin can be a promising, rapid modality for detection of hypercholesterolemia